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ABSTRACT 


Forensic art has been used for decades as a tool for law enforcement. When crime 
witnesses can provide a suspect description, an artist can create a composite drawing in 
hopes that a member of the public will recognize the subject. In cases where a suspect is 
captured on film, that photograph can be submitted into a facial recognition program for 
comparison with millions of possible matches, offering abundant opportunities to identify 
the suspect. Because composite images are reliant on a chance opportunity for a member 
of the public to see and recognize the subject depicted, they are unable to leverage the 
robust number of comparative opportunities associated with facial recognition programs. 

This research investigates the efficacy of combining composite forensic artistry 
with facial recognition technology to create a viable investigative tool to identify 
suspects, as well as better informing artists and program creators on how to improve the 
success of merging these technologies. This research ultimately reveals that while facial 
recognition programs can recognize composite renderings, they cannot achieve a level of 
accuracy that is useful to investigators. It also suggests opportunities to better design 
facial recognition programs to be more successful in the identification of composite 
images. 
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I. INTRODUCTION 


A. PROBLEM STATEMENT 

While valuable as an investigative tool, forensic imagery as a method of suspect 
identification is heavily reliant on the ability of the investigative agency to effectively 
distribute a particular sketch to the largest possible number of community members in 
order to find someone who can identify the wanted person. This method of circulation 
leaves the identification of the suspect somewhat to chance in that it relies solely on the 
possibility that someone who might know the identity of the wanted person would see the 
composite drawing. While the ability of investigators to generate forensic renderings of 
crime suspects is great due to the availability of artists and art programs, law enforcement 
officials are limited in their ability to circulate these renderings to a significant portion of 
the population. 

Comparatively, while scenarios involving photographs of suspects in the 
commission of a crime offer the best opportunity for suspect identification and eventual 
criminal prosecution, occasions in which a suspect is captured on film in the commission 
of a crime are somewhat rare. This infrequency limits the investigators’ opportunities to 
leverage facial recognition technology as an investigative tool for suspect identification. 

This juxtaposition illustrates the value of this research. While investigators, 
creators of facial recognition programs, and forensic artists may have an idea about the 
ability to successfully merge these two disciplines, it is not known whether composite 
drawings can be used effectively in combination with facial recognition software 
programs to identify unknown persons. 
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B. RESEARCH QUESTIONS 


1. Primary Research Question 

How can composite forensic imagery be used with facial recognition technology 
to increase the efficacy of forensic imagery as an investigative tool to identify possible 
persons of interest? 

2. Secondary Research Questions 

a. What can composite sketch artists learn from this research to improve the 
level of success by which their composite drawings contribute to the identification of 
persons of interest in a facial recognition software environment? 

b. What can creators of facial recognition software programs learn from this 
research to develop programs that recognize composite sketches more effectively and 
accurately? 

C. TENATIVE SOLUTIONS 

Facial recognition programs are abundant, varied, and utilized by numerous law 
enforcement agencies to identify criminal suspects captured by video or still 
photography; however it is not currently known whether this technology is a viable 
method to identify criminal suspects depicted in forensic composite images. Integrating 
composite art into facial recognition technology provides an opportunity not only to 
expand the suspect identification capabilities of investigators but also to measure more 
precisely the usefulness of merging these technologies. 

Sketch artists traditionally draw composite images by hand, but a new technology 
has emerged allowing less artistically talented individuals to complete composite images 
using a computer program. Images drawn by hand by forensic artists differ greatly in 
appearance from images created using a computer program. This research will also seek 
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to provide a better understanding of how the difference in the viability of each type of 
forensic image when entered into a facial recognition program will influence the efficacy 
of combining these two technologies. 

While investigators may have a more prolific resource of forensic renderings of 
crime suspects, they have a limited ability to circulate these renderings to a significant 
portion of the population. Conversely, many law enforcement organizations utilize facial 
recognition databases with millions of photographs; however, without a photograph of 
the suspect who committed the crime, they are powerless to identify that subject using 
facial recognition software. 

A possible solution to this problem is to explore the use of a facial recognition 
program to assist investigators in the identification of investigative leads depicted in 
composite images. This study will examine the merger of composite imagery with facial 
recognition technologies as a new opportunity to identify potential crime suspects. Facial 
recognition programs are abundant, varied, and used by numerous law enforcement 
agencies to identify criminal suspects captured by video or still photography, but it is not 
currently known whether this existing technology is viable for the identification of 
criminal suspects depicted in forensic composite images. 

D. SIGNIFICANCE 

This research will provide composite artists with information to assist them in 
creating images that are more precisely consumed by the facial recognition program, 
improving the program’s ability to identify possible investigative leads, and it will 
provide facial recognition program designers with information that will assist in possible 
design changes that better support the use of composite artistry in the future. 

The consumers of this research are forensic artists, facial recognition technology 
developers, and law enforcement, security, and investigative organizations at the 
municipal, state, federal, and international levels, both public and private. The 
intelligence community, criminal investigators, forensic artists, and the public regularly 
strive to identify those who may bring harm to our communities. 
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Government and law enforcement entities have invested billions of dollars in 
technologies that enable us to identify would-be terrorists and criminals through 
networked camera systems and facial recognition technologies. These programs enable 
law enforcement practitioners to identify threats and mitigate them in order to keep our 
communities safe. This research introduces a significant opportunity to expand the ability 
to identify criminals when no photographic representation exists of the individual 
committing the crime. These case studies will assess the merging of facial recognition 
technology and composite imagery to identify suspects in criminal investigations; it 
serves as one of the only known efforts to memorialize such information for future 
research. 

If successful, the use of facial recognition programs to identify a person depicted 
in a composite image may increase the ability of investigators to locate suspects and 
solve crimes. Additionally, this method would save investigator time, be less reliant on 
the ability of an investigator to circulate or show a composite drawing to a large 
population of people, and increase the potential for the use of forensic imagery to be 
successful in identifying suspects. In addition, it is likely that forensic artists and facial 
recognition creators alike will learn more about the ability to merge these two disciplines, 
improving their ability to more closely integrate these tools for increased success in the 
identification of unknown persons. This success will be measured in terms of the overall 
average effectiveness of one medium compared to the other. 
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II. BACKGROUND 


In lieu of a literature review, research in areas related to computer facial 
recognition and composite artistry is presented in this chapter. The overarching discipline 
of forensic artistry is discussed, followed by its four primary disciplines. This chapter 
highlights the important aspect of the forensic artist witness interviews and provides a 
baseline of information on the human memory process and its fallibility. This information 
provides a cursory understanding of the history, evolution, and future of composite art, 
composite imagery, and facial recognition technology. It illustrates the complexity of 
forensic art outside the parameters of creating an artistic representation and represents the 
full spectrum of the composite drawing process. 

A. FORENSIC ARTISTRY 

In today’s era of CSI (Crime Scene Investigation) television shows, “Forensic 
Files” documentary programs, and twenty-four-hour access to “Court TV,” the term 
“forensic art” can take on many meanings. The most common and well known of forensic 
art disciplines, composite imagery, is just one of several forensic art disciplines. To fully 
understand composite imagery, otherwise known as composite drawing, it is valuable to 
have a working definition of forensic artistry as a whole. 

1. Forensic Art 

Forensic art is any art used in conjunction with legal procedures or investigations. 
More specifically, forensic art is any art that aids in the identification, apprehension, or 
conviction of criminal offenders, or that aids in the location of victims or identification of 
unknown deceased persons (Taylor, 2001, p. 3). Forensic art is often multimedia in 
nature, with its primary purpose to present visual information. Even when the style of art 
and medium vary, all forensic art shares a common trait: it presents visual information 
(Stewart and Richlin, 1989). 
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Other aspects of forensic art include demonstrative evidence, which is usually 
used for case presentation in court, image modification and image identification such as 
video enhancement to improve the quality of photographic evidence, or the use of photo 
facial recognition programs for comparative identification, and reconstruction and 
postmortem identification aids, which usually involve the use of found human remains to 
create a representation of a deceased individual, either in two-dimensional or three- 
dimensional form. 

2. History and Origins of Forensic Art 

Forensic art is not a new discipline. It has been used in one form or another for 
decades to assist law enforcement in investigations and to portray criminal events. Early 
drawings published in local newspapers portrayed the accused criminal while sitting in 
the courtroom. Crime scene diagrams slowly became the accepted method of 
documenting the position of evidence at crime scenes, and as criminal trial proceedings 
have become more complex, investigators and prosecutors have come to rely on forensic 
artists to reconstruct the action at a scene, the collision of vehicles, the trajectory of 
projectiles, and almost anything else imaginable to illustrate elements of a criminal act. 
Forensic or composite imagery, the art of drawing faces of unknown persons, has been 
used to depict suspects, identify the dead, progress the age of a child to show what he or 
she might look like years later, and even to illustrate unique items of missing property. 

A great historic example of the use of forensic artistry and forensic imagery is the 
well-known London England case of Jack the Ripper, a serial murderer who killed 
women at random by mercilessly dismembering them with a knife. Of the volumes that 
have been written about Jack the Ripper and the literal death grip he held over the city of 
London in the late 1800s, very little has focused on the efforts of police and investigators 
who were obligated to capture the villain and end the reign of terror (Cozart, 2009). 
During the mid- to late 1880s, there was a frequent stream of bodies found in alleys and 
floating in the Thames River in the Whitechapel area of London. The London 
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Metropolitan Police and the City of London Police were very quickly confronted with a 
different kind of criminal behavior for the first time. These horrific, violent and gruesome 
crimes were murders without an obvious motive. 

In 1888, the field of police science was in its infancy. The so-called Jack the 
Ripper suspect may have been gruesome in his acts, but those same acts caused a 
revolution in the police sciences by forcing the police to develop new techniques and 
tactics for detecting unknown offenders, rather than simply laying hands on an obvious 
suspect (Cozart, 2009). It is thought that efforts attempting to solve these crimes led to 
many advances in the law enforcement investigative practice and technique of the time. 
Those advances include crime scene security, detailed inventories of evidence, and most 
significantly, numerous forms of documentation of various aspects of the crime. 

For the first time, detailed measurements plotted on a sketch of the area located 
the precise placement of every item found at the crime scene (Cozart, 2009). Drawings of 
the deceased victims, depicting the injuries both at the crime scene and at the time of 
autopsy, were completed graphically. In addition to these new forms of documentation, 
this investigation is thought to be the first time that composite imagery was used to depict 
the features of the suspect in a composite drawing in order to inform the community of 
the suspect’s appearance. 
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Figure 1. Jack the Ripper crime scene photo and related sketch. 

Source: http://commons.wikimedia.org/wiki/Jack the Ripper 



Figure 2. Artist depiction of additional victim at autopsy 

Source: http://photos.casebook.org/thumbnails.php?album=35 
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Figure 3. Early “composite” depictions of Jack the Ripper 

Source: http://en.wikipedia.org/wiki/Jack the Ripper 

In 1888, the police and the press made an attempt to garner public assistance in 
identifying Jack the Ripper. Two suspect sketches (accuracy unknown) appeared in the 
Illustrated Police News on October 20, 1888. Unfortunately, these sketches were deemed 
more the artist’s impression of what a sinister and evil Jack the Ripper should have 
looked like than accurate sketches, which could have helped identify the suspect. While 
these pictures likely boosted the paper’s circulation, they did little to catch the true killer. 
The killer was never identified and remained at large. Then, without explanation, the 
homicides ceased. 

Since the 1880s, suspect sketch artistry, now known as composite imagery, has 
improved greatly. Artists started paying more attention to details like head shape, hair 
length, and distance between eyes, nose, and mouth. The more sketches that were done, 
the more experienced artists realized that facial features could be broken into groups or 
sets, leading to the addition of the first attempts at automation of the process of creating a 
composite sketch. 
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In more recent history, the forensic imagery depictions of the “Unabomber” Ted 
Kaczynski in 1978 and the “Oklahoma City Bomber” Timothy McVeigh in 1996 
represent well-known examples of forensic imagery that offered law enforcement the 
only physical representation of then unknown suspects on a crime rampage. In both 
cases, forensic imagery was used effectively to pursue domestic terrorism suspects, and 
in the case of McVeigh, the image created by the FBI forensic artist bore an uncanny 
resemblance to McVeigh, enabling several witnesses to confirm his presence at the scene 
of the bombing and at the truck rental office days prior to the crime. 



Figure 4. Composite images of Unabomber Ted Kaczynski and Oklahoma City 

bomber Timothy McVeigh 

Source: http://en.wikipedia.org/wiki/Ted Kaczynski ; 
http://en.wikipedia.Org/wiki/FiIe:Aa McVeigh sketch and pic.jpg 

3. Forensic Artists 

Some forensic artists are trained in all of the forensic art disciplines, but more 
frequently forensic artists will specialize in one or two aspects of forensic art. Forensic 
artists enter the field of forensic art from many directions. In the case of forensic 
imagery, some are trained artists; others may be police officers that became interested in 
composite imagery out of necessity; and still others may be civilian employees of a law 
enforcement agency who have superior artistic and/or interview abilities. In the case of 
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those working in the field of image modification and facial recognition, these 
practitioners likely excel in computer skills and have a working knowledge of video- and 
photo-based programs. 

4. Art or Science? 

Many would argue that there is a requisite amount of science involved in forensic 
art. Most successful forensic artists collaborate with members of the scientific community 
for successful outcomes. This collaboration is most likely to occur in cases involving 
found human remains, but it is not limited to these types of investigations. Chemists, 
biologists, forensic anthropologists, and very often medical doctors of varied disciplines 
collaborate with forensic artists to piece together the details associated with a set of found 
human remains in order to determine the subject’s age, sex, race or ethnicity, height, and 
weight. These details are all valuable clues in determining the identity of the victim. 

B. COMPOSITE IMAGERY 

Composite imagery is most precisely defined as graphic images made up from the 
combination of individually described component parts (Taylor, 2001). Composite 
imagery can be used to depict objects, evidence, wanted persons, missing persons, the 
face of found human remains, and even the age progression of missing children and 
wanted or missing adults. To the casual observer, composite imagery is often thought of 
as the sketches of wanted suspects shown on the six o’clock news, but composite imagery 
takes on many forms. Besides the sketches with which we are all familiar, there are 
several other forms of two-dimensional composite imagery—as well as three- 
dimensional facial reconstruction—that are considered composite imagery. 

1. Composite Sketches 

Composite imagery and the composite sketch are the most commonly known of 
the four categories of forensic art. A composite image of a suspect enables law 
enforcement and the public to better focus on a suspect’s appearance based on victim and 
witness descriptions rather than on some erroneous image created in their own minds 
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based on their individual interpretation of a verbal or written description only. Knoxville 
Tennessee Detective Art Bohanan said: 

A picture is worth a thousand words, they say, but a composite sketch is 
worth 800—it gives you something to work with. The description of a 
white male, 5’8”, with brown hair doesn’t do anything for you, but if you 
can put some type of face with the description, then you can start looking 
for that person. (Mulholland, 2011). 

One of the most important concepts to consider before evaluating this research is 
the investigative and artistic intent of creating a sketch, or “sketch quality” composite 
representation of an unknown person. The benefit of a “sketch quality” drawing (versus a 
portrait quality drawing) in the process of identifying an unknown subject is that “sketch 
quality” in a composite rendering offers a key advantage over a more photographically 
detailed drawing. When viewing a completed composite sketch, one is forced to apply a 
very fruitful margin for interpretation rather than a narrow and unforgiving scope 
(Taylor, 2001, p. 202). 

While on one hand each artist likely strives for the perfect depiction of the 
described suspect in the context of artistic skill, the composite sketch is intended to depict 
a “likeness” of the suspect, not a precise portrait rendering. An effective composite 
sketch reminds its viewer of a few key elements of the suspect, or a noticeable detail, and 
not a complete and comprehensive portrait of the unknown person. The viewer of a 
composite sketch almost expects certain details to be off or imperfect. In this context, we 
must remember the difference between a portrait and a likeness. A portrait contains fine 
detail and is impossible to create without a photo or the actual subject of the portrait, but 
a likeness includes simple shapes. 

In the “likeness” depicted in Figure 5, for example, eyebrows and the slight 
widow’s peak in the hairline, as well as the overall shape of the face, nose, and eyes 
enabled the identification of this suspect. 
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Figure 5. Similar shapes depict noticeable features. Source: Phoenix Police 

Department. 

Composite imagery is often thought of solely as a method for identifying a 
wanted subject involved in criminal activity, but in the 1980s, law enforcement placed an 
increased emphasis on the problem of missing and endangered children (Taylor, 2001, 
p. 34). In the forensic imagery community, this resulted in a need for artists skilled in the 
techniques of age progression. With adult subjects age progression was usually 
accomplished through the updating of existing photographs. In the case of age 
progression of long-missing children, craniofacial growth data must be considered in 
order to account for facial evolution from childhood to adulthood. This process involves 
not only the study of the clinical evolution of human beings into adulthood but also the 
incorporation of familial traits into the renderings. The following is a twenty-year age 
progression of an adult homicide suspect. 
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Figure 6. Suspect at time of murder—twenty-year age progression composite— 
Suspect at time of arrest. Source: Phoenix Police Department. 

2. Composite Imagery-Automation and Computer Aided Methods 

For forensic imagery, one of the first attempts at automation of the composite 
sketch process came in the 1950s and was known as the Identi-KIT product; the similar 
British version produced years later was known as the PHOTO-FIT. The Identi-KIT used 
overlapping layers of clear plastic with various facial features printed on them (Cozart, 
2009). The combination of a series of layers would create a composite representation of 
what a suspect might have looked like, however the choices of varied facial details were 
few, and these limitations made a realistic depiction of the suspect difficult to create. 
Likewise, the PHOTO-FIT utilized a series of facial features organized in the center of 
the face with a facial shape surrounding them. 

The next evolution in forensic imagery was the development of computer 
programs that gave the nonartistic investigator an option in creating a forensic image of a 
crime suspect. Programs such as FACES and Photoshop produce programs designed 
especially for law enforcement to assist in creating composite images. Much like the 
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Identi-KIT, these programs are only limited by the number of choices and possible 
combinations that can be generated in the quest to create an image of a wanted subject. 

Very few police departments have a full-time forensic artist—in many cases 
shrinking budgets mean a lack of funds for such a position. For strapped law enforcement 
agencies with an infrequent need for a forensic artist, computer software is a low-cost 
alternative. A yearly fee of $50 or a one-time fee of $3990, for example, gives a police 
department or sheriff’s office access to thousands of facial features, hairstyles, tattoos, 
and other identifying marks—all of which are mixed and matched to create a composite 
(Mulholland, 2011). 

While these programs are a good second choice when a forensic sketch artist is 
not available, this technology does not replace a talented composite artist. Even the 
president of the Identi-KIT software company, Paul Wright, notes that no computer 
software is the same as a great forensic artist. “The truly good police sketch artists who 
know what they’re doing can make things on paper that we can’t in the sense that it (the 
drawing) is infinitely variable, because you have a human being creating it” (Mulholland, 
2011 ). 


C. RECONSTRUCTION AND POSTMORTEM IDENTIFICATION AIDS 

One of the most unpleasant and complicated tasks for law enforcement personnel 
is dealing with deceased bodies for whom no immediate means of identification can be 
found (Taylor, 2001, p. 303). There are two standard methods for postmortem 
identification: creation of a two-dimensional image—usually a composite sketch created 
from viewing the deceased or through the use of manipulation of X-rays of the skull—or 
creation of a three-dimensional image using the victim’s skull to re-create facial features 
of the victim. Both two-dimensional and three-dimensional reconstruction methods create 
a facial rendering capable of entry into facial recognition programs. 

Facial reconstruction from the skull is a very complex and detailed process. By 
comparing one skull to another, we are able to see the vast variability among people. To 
the trained eye of the anthropologist or skeletal biologist, the distinctions of race, 


ancestry, age, and sex are also usually determinable from the skull (Taylor, 2001, p. 362). 
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Both two- and three-dimensional methods of facial reconstruction have proven 
successful, but some cases may be more appropriate for one method over the other. 
Fragile or historic skulls may be too brittle to withstand the weight and pressure of 
applying clay, and skulls with facial damage may be compromised to the extent that they 
cannot support the application of clay accurately in damaged areas, lending more value to 
a two-dimensional reconstruction. Three-dimensional reconstruction offers the viewer an 
opportunity to see the re-creation from various perspectives and, in cases where personal 
belongings, clothing items, glasses, and jewelry were found, these items can be displayed 
with the three-dimensional representation. 

1. Two-Dimensional Reconstruction through Forensic Imagery 

When the remains of a deceased person are still in good enough condition to be 
recognizable, forensic artists may be able to draw a reasonable facial likeness of the 
individual. This is accomplished by either viewing crime scene photographs or, if 
possible, by viewing the actual body. This rendering will enable law enforcement 
officials to use the forensic image, rather than a photo of the deceased person, to attempt 
to identify the subject. Once a friend or family member identifies the deceased from a 
forensic re-creation, medical and dental records can be obtained to confirm the 
identification. 

Postmortem drawing is used in cases involving suicides, homicides, and natural 
deaths, as well as traffic fatalities and other blunt trauma deaths, gunshot fatalities, 
drowning, and nonextensive fires. 

2. Two-Dimensional Reconstruction through Image Modification 

In cases where decomposition, blunt trauma, animal activity, or other facial flesh 
disfigurement is present and re-creating the face is not possible through simply drawing 
the deceased, image modification may assist the forensic artist in creating an image of the 
unknown person. Two-dimensional reconstruction involves the artist’s use of the skull as 
a starting point for estimating facial features. 
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Tissue depth and various predetermined formulae for individual features are 
added to increase the artist’s understanding of the deceased’s facial structure. The skull is 
then photographed or radiographed (X-rayed) and tracings are made of both frontal and 
lateral views of the skull. The artist is then able to create a forensic image of the victim, 
based on the shape of the skull and through additional input from a forensic 
anthropologist regarding the locations of specific facial features such as the eyes, tip of 
the nose, and ear cavity position. 

If the forensic artist must resort to using the skull as a means for identification or 
formation of a composite representation, this method is the more cost effective of the two 
methods involving the skull, and it leaves the skull intact, undisturbed, and available for 
further examination. 

3. Three-Dimensional Reconstruction 

Three-dimensional reconstruction, sometimes called facial sculpture or skull 
reconstruction, is a method of forensic art used to identify skeletal remains. The artist and 
the anthropologist collaborate to construct the facial features of the unknown individual 
on the basis of the underlying cranial structure (Taylor, 2001). In situations where a 
deceased person is found in advanced or complete decomposition, the creation of two- 
dimensional drawings may not be an option for identification. In these cases, where the 
remains are skeletonized, a three-dimensional method of re-creating the facial features of 
the deceased may be the best opportunity to identify the subject. In a three-dimensional 
format, viewers of the completed re-creation are able to see the representation from 
various perspectives, giving the viewer a better opportunity to recognize the deceased. 

The fifty-five-year-old case of “Boulder Jane Doe,” for example, was solved with 
a 3-D reconstruction of the victim’s head. The victim’s niece recognized the victim from 
the 3-D rendering. 
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Figure 7. “Boulder Jane Doe” 3-D reconstruction. 
Source: http://www.dailycamera.com/news/ci 13658937 


D. DEMONSTRATIVE EVIDENCE 

Artistic representations presented as visual evidence can be used as demonstrative 
evidence in court. Using sophisticated computer programs, artists can enhance otherwise 
blurry videos and photographs, creating animated versions of the commission of the 
crime. By using global positioning satellite technology (GPS), artists are able to plot 
precisely the location of every piece of evidence at a crime scene, depicting the scene 
visually in ways that were never before imagined. Using GPS technology, points of 
investigative interest at complex traffic collisions can be plotted on a map, and 
investigators can visually experiment with varied scenarios to determine the cause and 
manner of the collision. These depictions can be used later at trial to depict complicated 
vehicle and crime scene dynamics in a simplified way, enabling juries to have a clearer 
understanding of what occurred. 

Demonstrative evidence may be presented as a slide show, PowerPoint 
presentation, a large diagram or other drawing, or even an enhanced still photograph. As 
technology advances, crime scene animation videos have become a more prevalent form 
of demonstrative evidence. Videos can depict the “action” at the crime scene, showing a 
car collision, an assault, the discharging of a firearm, blood spatter dispersal, and any 
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number of additional dynamic activities. They can illustrate building floor plans and large 
geospatial areas to enable a better understanding of large or complicated scenes. 

Such visual aids serve to explain and simplify complex concepts and provide 
understandable explanations for what may have occurred at a scene. In addition, animated 
depictions of rather gruesome evidence, or of deceased persons, can eliminate the shock 
to those viewing the evidence, minimizing the hardship to the loved ones of the deceased 
who may be present in court. Visual evidence of this nature has been used for several 
hundred years to depict crime scenes and autopsy results, as well as the location of items 
of evidence found at crime scenes. 

E. IMAGE MODIFICATION AND IMAGE IDENTIFICATION 

Image modification and image identification are methods of manipulation, 
enhancement, comparison, and categorization of photographic images (Taylor, 2001). 
While image modification has some relevance to this research, aspects of image 
identification, through facial recognition technology, play a large part in this study. 

1. Image Modification 

Image modification is a forensic art technique used to alter or enhance a 
photograph for the purpose of age progression, age regression, updating, or changing a 
subject (Mancusi, 2000). Image modification in the context of a criminal investigation is 
usually accomplished through one of two forensic art techniques. The enhancement can 
be drawn or painted on an image of the subject, or enhancements can be created digitally 
utilizing a computer and any number of specialized software programs. Age progression 
techniques are at times considered image modification. 

2. Image Identification—Facial Recognition Technology 

What is facial recognition technology? Humans have always had the innate ability 
to recognize and distinguish between faces, yet computers only recently have shown the 
same ability. Work on using computers to recognize faces originated in the mid 1960s 
(Bonsor & Johnson, 2008). Facial recognition from still images is one of the leading 
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image-based biometrics. For example, the U.S. Department of State operates one of the 
largest facial recognition systems in the world. In 2004, the original photo database was 
populated with 35 million visa photos. Currently (2011) the database contains over 75 
million photographs, integrating several Department of State photo resources. The 
database is actively used for visa processing and yields as many as 2,000 matches in an 
hour (Identix Coorporation, 2004). In applications where illumination and view angle are 
highly controlled and image resolution is high, recognition rates for the leading 
still-image facial recognition systems is very high (Schmitt et al., 2010, p. 1). 

What is facial recognition and how does it work? A facial recognition system is a 
computer application used to automatically identify or verify a person from a digital 
image or a video frame from a video source. Facial recognition programs are often used 
in security systems to assist in the identification of a person, utilizing a robust bank of 
photographs of known persons. They are also used extensively by law enforcement 
agencies to identify unknown subjects in various types of photographs. 

Every face has numerous landmarks, or peaks and valleys, which make up facial 
features. Much like locations of identification on a fingerprint, these facial landmarks, 
known as nodal points, are measured creating a numerical code. Each human face has 
approximately 80 nodal points, for example, the distance between the eyes, width of the 
nose, depth of the eye sockets, shape of the cheekbones, and length of the jaw line. 

The most traditional facial recognition technique is identification of persons 
through the use of a mathematical algorithm designed to identify specific landmarks or 
features from an image of the subject’s face. There are two types of applications of this 
algorithmic method, geometric algorithms and photometric algorithms. 

a. Geometric Algorithm 

An algorithm expressed geometrically may identify the relative position, 
size, and/or shape of the eyes, nose, cheekbones, and jaw forming a mathematical 
representation for this geometric relationship between features. 
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Figure 8. Geometric algorithm. 


Source: 

http://www.google.com/search?q=facial+recognition+geometric+algorithm&hl=en&clien 

t=safari&rls=en&prmd=ivns&source=lnms&tbm=isch&ei=d4VrTsfOKpPKiAKv- 

ryqDg&sa=X&oi=mode link&ct=mode&cd=2&ved=0CAwQ AUoAQ&biw= 1224&bih 

=650 


b. Photometric Algorithm 

An algorithm is expressed photometrically when these facial features in 
combination are given a specific numerical representation based on distilling an image 
into numeric values and comparing the values to other values. 


21 








Figure 9. Photometric algorithm 

Source: 

http://www.google.com/search?q=facial+recognition+geometric+algorithm&hl=en&clien 

t=safari&rls=en&prmd=ivns&source=lnms&tbm=isch&ei=d4VrTsfOKpPKiAKv- 

ryqDg&sa=X&oi=mode link&ct=mode&cd=2&ved=OCAwO AUoAQ&biw= 1224&bih 

=650 

c. The Facial Recognition Program Used for This Research 

There are many facial recognition products on the market available to law 
enforcement, all with varied features and searching methods. The facial recognition 
program utilized in this research is the Image Investigation System (IIS), which was 
developed by Hummingbird Defense Systems, a Scottsdale, Arizona-based company. 
This system uses a geometric algorithm, focusing on 17 geometric facial relationships, 
and it is equipped with case tracking (regulated management of intelligence information) 
that is compliant with the U.S. Code of Federal Regulations, Title 28 (CFR28). This 
particular program starts with a focus on the eyes and examines their relationship to other 
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areas or points on the face creating the geometric algorithmic representation of each 
image. The program does not use age, sex, race, or ethnicity in its search criteria. 

The IIS program is housed at the Arizona Counter Terrorism and 
Information Center (ACTIC) located in Phoenix, Arizona, and is operated by the 
Maricopa County Sheriffs office. Two hundred forty state, local, and federal 
investigators, analysts, and support personnel from forty-three different agencies are 
assigned to the center, including the FBI Joint Terrorism Task Force (JTTF) and the Field 
Intelligence Group (FIG), conducting both criminal and intelligence investigations. The 
IIS is regularly used by these agencies to solve crimes. 

The number of images in the database exceeds 30 million, and IIS can 
provide search capabilities for photographs, video, and surveillance still photographs of 
wanted suspects and known or suspected terrorists. The IIS database is constantly 
supplied with new photographs from the Arizona Motor Vehicle Department (driver’s 
license and state ID cards), mug shots from Arizona jail bookings since 1998 (4.5 million 
photos currently), the U.S. Justice Department’s Joint Automated Booking System 
(JABS), and the state’s sex offender database, consisting of over 12,000 pictures. The 
ACTIC facial recognition unit is capable of comparing a single image to the 30 million 
photos in its database in 15 seconds or less (Bristow, 2010). 

d. Processing a Photograph in IIS 

The process for comparative analysis of an unknown offender photograph 
using the IIS consists of “enrolling” the pictures of unknown suspects into the program. 
The program then uses the geometric algorithm to create a numeric representation of the 
face of the unknown person. Once this numeric representation is determined, it is 
compared against the numerical representations of all the photographs of known subjects 
in the database. Within 15 seconds, the program delivers a list of possible matches. It also 
stores the image of the unknown person and compares it to new photographs entered into 
the system subsequent to this analysis, affording investigators the ongoing potential to 
identify the suspect. 
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i. Identification of Possible Matches 

The possible matches identified by the system are then reviewed 
by a Facial Recognition Unit detective who visually compares the picture of the unknown 
suspect with the pictures associated with the possible matches to determine which 
photographs warrant further investigation by the case detective who submitted the 
unknown suspect photograph. Because this program does not use age, race, ethnicity, or 
sex to identify possible suspects, the Facial Recognition detective will eliminate possible 
suspects that are obviously not the same sex, race, or close enough in age to the suspect 
picture being enrolled. 

ii. Inconsistencies in Photographic Standards 

This visual review is also necessary because, despite the high 
performance of still-image systems in controlled conditions, performance can drop quite 
rapidly as variations in view angle, illumination, occlusion, or viewing distance increase 
or as image resolution decreases (Schmitt et al., 2010, p. 2). Essentially, all photographs 
are not taken with the same levels of focus, lighting, angle, distance, and facial 
perspective. Subjects who take photographs of themselves in a mirror also present a new 
variable, requiring the photograph to be reversed before it is enrolled into the database. 
These variables can cause slight differences in the photographic representations of the 
subjects. While the inputting of the photographs into the program is a quick and easy 
process, the visual review process is time intensive. The ACTIC Facial Recognition Unit 
processes an average of 9000 individual comparisons per month (Bristow, 2010). 

iii. Programmatic Range 

The Facial Recognition Program has the ability to search at various 
levels of inclusion. These levels of inclusion translate to a percentage of certainty or 
possibility. If a search is conducted at a high level of certainty, 95% for example, the 
program may identify a small number of possible matches, but due to photographic 
differences between the subject in the commission of the crime and the photos in the 
database, searching at a high level of certainty might unintentionally eliminate the true 
photographic match from the list of possible suspects. Conversely, searching at a low 
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level of certainty, 50% for instance, could yield an unwieldy number of photographs, 
requiring the reviewing detective to view tens of thousands of photographs to identify a 
match, a process that is not practical. 

The standard search percentage for the ACTIC Facial Recognition 
Unit is 90%. At this percentage the program is likely to identify approximately 500 
possible matches for visual review. This number is considered productive for the 
furtherance of most cases accepted for facial recognition program searches. This standard 
will be used in the case study to determine the usefulness of composite picture hits in the 
database. 

F. THE INTERVIEW PROCESS AND THE PITFALLS OF EYEWITNESS 
DESCRIPTIONS 

In preparing a high-quality composite image, the interview is everything (Taylor, 
2001, p. 137). The ability to successfully and effectively interview witnesses and victims 
of crimes is key to creating the most accurate forensic image possible. Many 
psychological and medical studies have been conducted regarding human cognition and 
perception, the impact of trauma on memory, facial recognition, eyewitness 
identification, and interview techniques. These elements and many more have direct 
bearing on the work of a forensic artist. As if it is not difficult enough to deliver an 
accurate artistic representation of a person of interest, forensic artists must also know 
about the ability or inability of human beings to remember. 

1. The Interview 

At some point during any investigation, there will come a time to interview and 
question people. There are unique differences between interviewing a witness and 
interrogating a suspect (Cozart, 2009). The process of creating a composite drawing 
involves the artist’s skillful interview of an eyewitness to the crime, followed by the 
creation of an artistic rendering of the described suspect. A witness can be a knowing 
observer of the crime, a victim, and even an unwitting bystander, unaware of the crime 
afoot. These observers are often asked to describe the suspect of the crime with enough 
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detail that a skilled forensic artist can draw a likeness of the suspect’s face. Although 
recognizing people by their faces is something people commonly do without having to 
think about it, the processes involved in face recognition are far from simple (Nickoh, 
2003). 

While few, if any, forensic artists or their renderings have been challenged in 
court, the descriptions provided to the artist by those present at the scene are frequently 
questioned. In order to maximize their effectiveness, forensic artists must understand the 
human memory process and be cognizant of the issues that can reduce the reliability of 
eyewitness descriptions. This knowledge allows the forensic artist to give consideration 
to the use of specific interview techniques that minimize the opportunities to garner 
inaccurate accounts or descriptions of crime suspects and maximize the forensic artist’s 
ability to create an accurate rendering of the suspect. 

The police composite artist, whose function is to develop an image based on an 
observer’s recollection of a person or event, must have a strong knowledge of the 
dynamics of human memory. This enables the artist to more effectively retrieve the 
information needed to create the composite drawing. Forensic art experts, as well as 
psychologists, describe this interview as a process in which the composite forensic artist 
balances common sense and experience with relevant theories about memory (Domingo, 
1989). Some artists and police agencies make the mistake of believing that strong 
drawing skills are the only criterion for productive composite drawing. In fact, a 
mediocre artist with superior interview and people skills is preferred over a skilled artist 
with mediocre interview skills. The ideal composite artist possesses strong drawing skills 
as well as strong interview skills (Taylor, 2001, p. 138). 

The paradox of eyewitness testimony is that, although justice professionals regard 
it as having a low level of reliability, it is the most important form of testimony as far as 
jurors are concerned (Nickoh, 2003). Although juries and decision makers place great 
reliance on eyewitness identification, they are often unaware of the potential danger of 
false memories and misidentification. In this context, the purpose of the composite 
drawing can be thought of as a quest for absolute truth. Absolute truth is reflective of 

reality, not someone’s personal view of reality. 
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In creating a likeness of a suspect, the artist focuses on identifiable features of the 
subject, the first of which is the shape of the head, followed by unique or distinguishing 
features such as ears, facial features, and the relationship of facial features, to other facial 
features (distance, shape, angle, and position to name a few). It is these fine details that 
must be recalled with accuracy in order to create the appropriate likeness of the wanted 
person. The most important thing that must be remembered is that no artist, no 
composite, and no computer can provide an accurate picture without a witness who can 
provide a good description (Cozart, 2009). 

Once finished, the composite drawing is not intended to be a portrait of the 
suspect but instead is designed to exhibit a likeness such that it will prompt those who 
view it to be reminded of specific features and feature relationships that may be similar to 
someone they know. Effective composite drawings may not completely represent the 
subject, but they accurately represent key details of the subject. 

To further explore the link between an accurate forensic representation and a 
reliable description provided by those present at the scene of the crime, it is important to 
understand how human beings observe and remember what they have seen. In addition, 
how these witnesses are questioned has further bearing on the accuracy of the 
information garnered by the interviewer. 

2. Original Memories 

Many experts believe the concept of an “original memory” is a fallacy because 
the process of interpretation occurs at the very formation of a memory—thus introducing 
distortion from the very beginning (Engelhardt, 1999). From that point forward, the 
memory is subject to further opportunities for distortion. It is widely known that 
witnesses can distort their own memories without the help of a third party. This is 
because people rarely tell a story or recount events without a purpose. The act of telling 
and retelling a story is tailored to a particular listener. In telling a story, we edit out some 
details and emphasize others. The proverbial “fish story” is an example of this 
phenomenon in which the teller of the story changes the story with each telling and 
ultimately ends up believing the evolving story. 
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Many people become committed to their perceived original memory as a matter of 
personal pride or to avoid embarrassment. This is often a product of the subconscious 
mind and not necessarily a deliberate process. Once a witness states facts in a particular 
way or identifies a particular person as a perpetrator, they are often unwilling or unable— 
due to the reconstruction of their memory, to reconsider their original understanding or 
interpretation. When a witness identifies a person in a line-up, he is likely to identify that 
same person in later line-ups, even when the person identified is not the perpetrator or 
when the real perpetrator is present. 

3. Fallibility of Witness Memories 

Further complicating the ability of a forensic artist to gather accurate suspect 
descriptions are simple mistakes in memory. Memory errors fall into two classes: people 
can either completely fail to recall an event, or they may have an inaccurate recollection 
of the event (Green, 2009). Most people have very different attitudes about the two types 
of failure. Most understand that total memory failures are common. They can reflect 
about occasions when they have been unable to recall an event, so failures by other 
people are hardly surprising. In contrast, people are often overly optimistic about the 
accuracy of their retrieved memories, mostly because these errors have little practical 
consequence and often go unnoticed. This enables most people to become committed to 
the accuracy of their memory. Given the overconfidence in their own memory, people 
often place too much faith in the accuracy of eyewitnesses. 

What causes our memory to be inaccurate? There are many reasons, but they fall 
into six basic categories: blurred memory, filling in for memory gaps, systematic 
distortion of perception, personal influences to memory, memory changes over time and 
with retelling, and finally—and most relative to the topic of forensic art—memory biased 
by question retrieval method. A skilled forensic artist is likely aware of all these 
impediments to accurate memory recall; however, combating them or adjusting for them 
in the interview process is a difficult task. Artists must consider the following potential 
pitfalls: 
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a. Blurred Memory 


Images in our mind’s eye are never as clear as the actual perception. For 
instance, everyone is familiar with their own bedroom, its general shape and size, as well 
as contents, but most people are at a loss to describe their bedroom in detail simply 
because they do not possess a memory of the room that is as detailed as actually looking 
at the room. Color is another example of the low resolution of human memory. There are 
thousands of colors, but color memory is very limited as to the shades of color within 
specific known colors. We are all familiar with ten to fifteen main colors, but shades of 
these colors can remain elusive to our memory. Finally, people do much better at picking 
between two objects when both are present, rather than when one is present and one is 
absent. This is important as it relates to viewing a live or photographic line-up of possible 
suspects. If the photo of the suspect or the suspect himself is not contained in the line-up, 
people may be inclined to choose from what they see, rather than rule out all possible 
suspects in the line-up in favor of a suspect who is not present in the array. 

b. Memory Fills in the Gaps 

Memory should be thought of as a reconstruction, not a record (Green, 
2009). Eyewitnesses will often have insufficient information in the memory itself, so the 
reconstruction of the memory must invoke pieces of information from other sources. 
These sources can be preexisting schemas or other memories. People often understand 
their world through specific schemas or scripts—these are stereotyped mental models of 
objects and events. People can also confuse information sources. Memory source 
confusion, that is, including separate memories from other events in a more recent 
memory, is very common. 

Psychologists have long recognized that gap filling and reliance on 

assumptions are necessary survival skills to function in our society (Engelhardt, 1999). 

For instance, if we are traveling on a crowded or loud subway, we might hear garbled 

words like “next,” “transfer,” and “train.” Building on our personal experience, 

knowledge, and assumptions, we might put together the actual statement: “Next stop 53 rd 

Street, transfer available to the F train.” If asked later, we might even “remember” that 
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we had heard the full statement from the beginning, even if we had not. This is a 
subconscious process that takes place daily as a matter of survival in the world. 

c. Memory Systematically Distorts Perception 

Memory tends to distort perception in systematic ways. People tend to 
remember colors as being brighter and more saturated than they actually were. Studies 
have shown that people who are asked to recall vehicle speeds tend to overestimate slow 
speeds and to underestimate fast ones. Memory is also biased toward expected events. If 
someone witnesses the beginning of a conversation in which the communicator is giving 
directions to a destination familiar to the witness, the witness may not hear the 
inaccuracies given in the directions because he or she knows the proper directions and 
expects them to be told accurately. 

d. Memory Is Personal 

Human memory does not exist so that an observer may accurately report 
previously seen events. Each witness extracts an interpretation that is meaningful in terms 
of his personal beliefs, experiences and needs. Once the interpretation occurs, the events 
themselves become relatively unimportant. This explains why multiple witnesses to the 
same traffic collision can report a different account of what occurred between the 
vehicles involved. Witnesses may report different colors of vehicles, direction of travel, 
and even different colors of traffic signals, and some may base their description on their 
understanding of relevant traffic laws, rather than what they really observed. 

e. Memory Changes over Time and with Retelling 

Numerous studies have revealed that memory can change significantly 
over time. Most commonly, witnesses incorporate information learned after the event into 
their memory. This information may be provided by other witnesses and is usually used 
subconsciously to fill in memory gaps. In addition, as people recall events over and over, 
they eliminate some details and add others. Forensic artists are often challenged to avoid 
providing a witness with additional information relevant to the crime, and many artists 
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avoid being briefed on the events surrounding the crime so that they are sure to gather the 
information provided only by the witness. 

/. Memory Is Biased by Question Retrieval Method 

Memories can be biased by the questions asked at the time of retrieval. 
Several well-known studies have shown just how much the question itself can provide 
information that witnesses can incorporate into their memory. Eliminating leading 
questions is key to a forensic artist’s interview process. 

Studies have been conducted on human memory and on subjects’ 
propensity to remember erroneously events and details that did not occur (Engelhardt, 
1999). Oftentimes the main cause of this deviation in memory is caused by a third party 
either intentionally or unintentionally introducing false facts into memory. In one study, 
conducted in the 1970s by Elizabeth Loftus, subjects were shown a film of a traffic 
collision and then asked questions about it. One question was “How fast were the cars 
going...?” Half were asked, “.. .when they hit each other?”, and the other half were 
asked, “.. .when they smashed into each other?” Subsequently, when asked to estimate 
the speed of the collision, the “smashed” group estimated higher speed than the “hit” 
group. Amazingly, when asked if there had been any broken glass related to the collision 
in the film (there had not), the “smashed” group was more likely to “remember” having 
seen broken glass. This suggests that the schema being used to reconstruct a memory is 
influenced by the way questions are posed. The “smashed” group was confabulating, that 
is to say, they honestly believed that they had seen broken glass. 

Successful forensic artists acknowledge at the onset of an interview that 
human memory does not operate as if it were a videotape camera. We do not simply 
record some event in our memory and then later retrieve an unblemished recollection of 
what happened. Human memory is much more fragile, suggestible, and prone to 
distortion and decay than we typically realize. As a result, mistaken eyewitness testimony 
rarely involves outright lies; instead, it usually corresponds to commonly occurring 
distortions in memory functioning. 
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After witnessing an event, people are sometimes exposed to new 
information that can actually change their memory. This postevent information effect 
often transpires as a result of dialogues with other people or in an effort to fill in the 
memory blanks and even in an effort to piece together a sequence of events that make the 
witness’s observations more relative. 

Because forensic artists are completely reliant on witness accounts of 
events, an awareness of these impediments to accurate memory recall is helpful as they 
move through the interview process in search of the most accurate information to create a 
facial image of the suspect. 

G. CONCLUSION 

While the literature provides a wealth of information on forensic imagery, facial 
recognition technology, and the investigative practices associated with both, the 
combined use of forensic imagery and facial recognition technology to identify unknown 
subjects of a criminal investigation is not well understood. This section will explore, in 
part, what has been accomplished to answer the question “How can composite forensic 
imagery be used with facial recognition technology to increase the efficacy of forensic 
imagery as an investigative tool to identify possible persons of interest?” It will provide a 
brief summary of what remains to be addressed. 

Upon examination of the evolution of forensic imagery, the literature suggests 
that much of this artistry is based firmly in what artists have come to know scientifically 
about the human face and facial structure and how the underlying skeletal structure 
influences the appearance of the complex expressions that make up the face. The features 
that make up a face are numerous, some obvious and some subtle, yet all are important 
when attempting to represent the overall depiction of the person described to the artist. A 
detailed knowledge of anatomy, science, psychology, communication, and interview 
techniques are all necessary skills for the composite artist. 

The research reveals not only the necessity for forensic artists to accurately 

document the person depicted in his or her representation but also the need for the artist 

to exploit certain memorable or pronounced facial features so that the drawing can help 
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its viewer better identify the unknown person it depicts. This can be thought of as the 
distinctiveness of proportional arrangement of facial features. This distinctiveness forms 
the subject’s “look” and triggers recognition (Taylor, 2001). To this end, composite 
drawings by nature should only contain a given amount of detail based on the limits of 
the human memory, focusing on the most pronounced features. This is a stark contrast to 
a photograph or an artist’s portrait of a subject seated before him or her, with every detail 
of the subject’s appearance. 

As we compare the art of forensic imagery with the technology of facial 
recognition, the literature reveals two methods for mapping facial structure 
mathematically. In the same way the forensic artist documents the relationship of facial 
features in a sketch, facial recognition technology views a photograph of a subject and 
expresses these facial relationships numerically. Whether it is photometrically or 
geometrically, the algorithmic representation of the face can be compared with other 
representations in an effort to identify similarities. A computer program conducts this 
comparison, and its results are the sterile outcome of mathematical algorithmic formulae. 
This process, however, is not completely free of human interpretation. Research suggests 
that this process is completed though human review of possible matches to help confirm 
the likelihood of identification of a true match. 

The literature available on forensic imagery and facial recognition suggests that 
there are similarities in both processes. Both use the relationships of facial features to 
create and identify a representation that brings about recognition from the viewer. The 
literature also documents differences in these processes. Initially, facial recognition 
programs are scientific, precise, and free from human interpretation. Forensic imagery is 
quite the opposite, utilizing interpretation, communication, inference, and varied levels of 
artistic ability to create a representation. It is these similarities and differences in forensic 
imagery and facial recognition that help to formulate the question “How can composite 
forensic imagery be used with facial recognition technology to increase the efficacy of 
forensic imagery as an investigative tool to identify possible persons of interest?” 

We have yet to learn whether these two very different methods of identifying 

human faces are suitable for comparison with each other. Does the creation of a sketch of 
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the human face lend itself to accurate interpretation of facial relationships 
mathematically? Does the forensically drawn image of a person express similarities in 
facial relationships in the same manner as a facial recognition program? Can a facial 
recognition program consume forensic images of human faces in a way that identifies 
possible suspects? Can the facial recognition program do so productively, and if it cannot 
be done at a level considered productive, what can composite sketch artists learn from 
this research to improve the level of success by which their composite drawings 
contribute to the identification of persons of interest in a facial recognition software 
environment? Conversely, what can creators of facial recognition software programs 
learn from this research to develop programs that recognize composite sketches more 
effectively and accurately? 
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III. METHODOLOGY 


A. METHODOLOGY 

Case study methodology was used for this thesis and consisted of a comparative 
analysis of several cases. Each case consisted of a group of composite sketches 
completed by a single artist. This research served to evaluate the efficacy of merging two 
previously separate identification technologies, composite art and photo facial 
recognition. 

This methodology helped to identify how composite imagery can be used with 
facial recognition technology to increase the efficacy of forensic imagery as an 
investigative tool to identify possible crime suspects. The methodology also provided an 
opportunity to obtain information that is beneficial to both composite artists and facial 
recognition program designers who may continue to pursue this merger of their 
individual disciplines. Ultimately this research may aid future researchers in further study 
of these two identification disciplines. 

The IIS Facial Recognition System used by the Arizona Counter Terrorism 
Information Center contains only photographs. Until now, composite sketches have not 
been entered into this system to test the ability of the technology to recognize a picture 
created in a medium other than photography. The primary media for hand-drawn 
composite sketches are pencil, pen, and charcoal. Composite images can also be created 
using a variety of software programs available to law enforcement. This research offered 
the opportunity to analyze the ability of the Facial Recognition Program to consume 
composite representations in both hand-drawn and electronically generated formats. 

Throughout this research the terms “composite sketch or composite art,” “mug 
photo,” and “portrait style sketch” or “control sketch” will be used. Although these terms 
may be used interchangeably in casual discussions about this topic, it is important to note 
their differences in the context of this research. 
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Composite Sketch or Composite Art —The composite sketch, whether drawn by 
hand or through the utilization of a software program, is an artistic rendering of a wanted 
person created by the artist immediately following the commission of a crime using 
witness and victim descriptions. This rendering is created before authorities know the 
identity of the suspect and is intended to be a “likeness” of the wanted person, not a 
portrait. 

Mug Photo —A mug photo is a photograph of a suspect taken at the time of arrest 
or incarceration. In the context of this research, the photo recognition database of mug 
photos also contains all Arizona drivers’ license photos. 

Portrait Style Sketch or Control Sketch —For purposes of this research, portrait 
style sketches of suspects were made utilizing the suspect’s mug photo as a model. The 
goal of these sketches was to try to duplicate the depiction in the photograph exactly. 
These drawings were utilized as a control sample to establish the ability of the Photo 
Recognition Program to accurately identify the suspect’s mug photo from the portrait 
style drawing of the suspect depicted in the mug photo. 

In addition to these terms, it is important to note the general method by which 
photographs, and now composite sketches of unknown persons, are entered into the facial 
recognition program in search of a matching photograph and ultimately the suspect’s 
identity. The process used to enter or “enroll” composite sketches into facial recognition 
followed the same procedure that is used to enroll photographs of unknown suspects. The 
process for comparative analysis of an unknown offender photograph using the IIS Facial 
Recognition Program starts with entering or enrolling the picture of an unknown suspect 
into the program. 

Upon loading the photograph database, the program analyzes the photo and 
views it in the form of a geometric representation. This geometric representation is used 
to create an algorithm or numeric representation of the face of the unknown person. Once 
this numeric representation is determined, it is compared against the numerical 
representations of all the photographs of known subjects in the database. 
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The Facial Recognition Program has the ability to search with varied levels of 
inclusion, which are expressed in the form of a percentage. The percentage reflects the 
level at which an identifiable number of photographs are excluded from the search. For 
instance, a search at 90% exclusion reveals only the top 10% of best possible matches to 
the enrolled photograph, which may consist of 200-500 possible matches. A search at 
40% exclusion reveals the top 60% of best possible matches and could yield millions of 
possible suspect photos. The higher the percentage of inclusion, the smaller the number 
of possible matches will be delivered. 

Within 15 seconds of entering the photograph, the program delivers a list of 
possible matches. It also stores the image of the unknown person and compares it to new 
photographs entered into the system subsequent to this analysis, affording investigators 
the ongoing potential to identify the suspect. Ideally, a search resulting in no more than 
500 possible matches is preferred. Subsequently, a program administrator visually 
reviews these matches looking for subjects who are obviously the incorrect age, race, or 
gender, thus reducing the number of possible matches delivered to the investigator for 
follow-up investigation. 

1. Fact finding inquiries conducted prior to the analysis of cases 

Three inquiries or tests preceded the examination of cases and were conducted to 
better understand how the facial recognition program would view composite sketches and 
how successfully the composite sketch might be utilized effectively with the facial 
recognition technology to identify suspects. In addition to testing the viability of this 
research, these inquiries also contributed to the author’s understanding of how to 
organize and structure the analysis of the cases. 

Prior to examining implementation of the actual case data related to the merging 
of composite imagery with a facial recognition program, it was necessary to determine 
whether the Facial Recognition Program could recognize a pencil medium “portrait style” 
sketch drawn from a mug photo as an image of a human face (Fact Finding Inquiry 
Number 1). Once this was determined, it was necessary to know whether the Facial 
Recognition Program could use this pencil medium “portrait style” sketch drawn from a 
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mug photo to locate the suspect’s mug photo itself within the Facial Recognition Program 
database (Fact Finding Inquiry Number 2). If the technology did have the ability to 
recognize a portrait style pencil drawing, it was then necessary to determine whether it 
could also recognize a composite sketch (Fact Finding Inquiry Number 3). These fact 
finding inquiries are detailed below. The analysis and results will be found in subsequent 
chapters. 


a. Fact Finding Inquiry Number 1: 

Determination of the IIS Facial Recognition Program’s Ability to 
Recognize a Composite Pencil “Portrait Style” Drawing as a Facial 
Representation 

The purpose of this inquiry was to determine whether the Facial 
Recognition Program was able to recognize a facial sketch drawn in pencil medium. To 
test this aspect of the program, a forensic artist obtained the mug photo of a suspect who 
was arrested in a case in which he had completed a composite drawing of the then 
unknown suspect at the time of the crime. Utilizing the mug photo, which had already 
been entered in the Facial Recognition Program database, the artist drew a pencil 
“portrait style” drawing of the suspect from the mug photo. This generated a portrait 
quality representation of the suspect as he appeared in his mug photo, rather than the 
likeness created through the original composite drawing. 

This pencil drawing was enrolled into the Facial Recognition Program to 
determine whether the program would recognize the drawing as a face and assign a 
numeric value to the picture using the geometric algorithm. Upon visual inspection of the 
sketch by the facial recognition system manager, he determined that the overall grey scale 
of sketches was too light, with very slight differentiation in features. Based on this 
feedback, the artist revised his sketch; darkening some of the lines in the area of the eyes, 
nose, and mouth to ensure that the features were recognizable (this program focuses on 
the eyes of a subject and the relationship of facial elements to the eyes). This revision did 
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not change the details in the portrait; it simply darkened them. The amended “portrait” 
sketch was entered into the Facial Recognition Program for comparison. (See Appendix 
A-l.) 


b. Fact Finding Inquiry Number 2: 

Determination of the IIS Facial Recognition Program’s Ability to 
Identify the Mug Photo of a Subject Using a Pencil “Portrait Style” 
Drawing of the Subject in the Mug Photo 

The purpose of this inquiry was to determine whether the algorithmic 
representation of the pencil drawing created by the facial recognition software in inquiry 
number 1 could generate a possible match to the mug photo of the subject depicted in the 
drawing. To answer this question, the numeric representation of the “portrait style” 
drawing generated in inquiry number 1 was entered into the Facial Recognition Program 
for comparison with the photos in the database in an attempt to locate the mug photo used 
to create the sketch. This is the same process used with any photograph being enrolled 
into the database in an attempt to locate a suspect. 

The Facial Recognition Program has the ability to search with varied 
levels of inclusion. This search for the suspect’s mug photos utilizing the pencil “portrait 
style” sketches of the mug photos was done first at a low degree of certainty and then 
increased to a higher level of certainty. Ultimately a 90% certainty level was used to 
locate a possible suspect. This is the same standard of certainty used when comparing 
photographs of unknown persons to photographs in the database. (See Appendix A-2). 

c. Fact Finding Inquiry Number 3: 

Can the IIS Facial Recognition Program Identify Potential 
Investigative Leads Suspects Using a Composite Drawing of the 
Suspect? 

This inquiry used the original composite sketch of the suspect depicted in 
the “portrait style” sketch in inquiries 1 and 2. The forensic artist had completed this 

sketch immediately following the crime. This composite was created using witness and 
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victim descriptions obtained in a standard composite imagery interview. The purpose of 
this inquiry was to determine whether the algorithmic representation of the pencil 
composite drawing (not the “portrait style” drawing) created by the Facial Recognition 
software could generate a possible match to the mug photo of the subject depicted in the 
drawing. A composite image is intended to be a likeness of the subject in question, not a 
portrait, making this an important inquiry. 

Having previously isolated the ability of the Facial Recognition Program 
to recognize a pencil medium “portrait style” drawing of the same suspect created by the 
same artist, this inquiry allowed a separate understanding of the ability of the Facial 
Recognition Program to recognize a pencil medium composite sketch of the suspect. This 
analysis allowed for comparison between the two results both visually and quantitatively. 
(See Appendix A-3.) 

d. The Case Studies 

Each case consisted of a group of composite sketches completed by a 
single artist. The composite images of each of the suspects were enrolled into the Facial 
Recognition Program in order to determine the level of inclusion at which the suspect’s 
mug photo would appear. This level is expressed in the form of a percentage. Through 
this percentage we are able to understand the level of effectiveness for each composite 
drawing. These cases were aggregated in total, per artist, and per artistic medium in order 
to understand the efficacy of merging these technologies. 

B. SAMPLE 

This research consists of four cases. Each case was associated with a single 
forensic composite artist and his or her composite artistry. Because this analysis required 
the ability to compare composite representations of suspects with actual mug photos of 
the suspects, only composite images with identified suspects were studied. Due to this 
constraint the number of individual composite representations examined per artist was 
dependent upon the number of representations associated with identified suspects. 


40 



The artists submitted the composite drawings and mug photos of all their viable 
cases in which a suspect had been identified. The analyses of these cases answers the 
stated research questions by providing data regarding the ability of the facial recognition 
system to identify a suspect through enrollment of a composite image of the suspect into 
the Facial Recognition System. Using the standard for successful photographic entry into 
Facial Recognition, a comparison can be made to understand whether that standard can 
be met in the use of composite drawings. 

All the cases are similar in that they are associated with a composite drawing of a 
person of interest associated with a real crime, documented through the forensic 
interview process. They were all enrolled into the same facial recognition software 
program, the sole program used by law enforcement in the state of Arizona. The variables 
associated with this process include different artists (with varied artistic and interviewing 
skill levels), the number of investigative cases used per artist, and the drawing medium 
used to create the composite (hand-drawn sketch or computer generated composite). 

In an effort to provide a visual comparison of a “portrait style” drawing to a 
composite drawing, the first sixteen composite art samples tested included a control 
drawing of the suspect created from the suspect’s mug photo. The display of the 
composite drawing, the mug photo, and the control or “portrait style” drawing allows a 
qualitative evaluation by the reader of the visual accuracy of the composite sketch 
compared to the accuracy of the “portrait style” control sketch. While it was not 
necessary to replicate the inclusion of this control drawing with all the composite 
drawings, analysis of these sixteen cases including the control drawings provides data 
comparing the ability of the facial recognition system to identify a suspect through 
enrollment of a “portrait grade” drawing of the suspect (used as the control drawing) 
created by the artist for this analysis, against the system’s ability to identify the same 
person through the enrollment of a composite image of the suspect drawn at the time the 
crime occurred. (See Appendix B-l through B-16.) 
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The cases are as follows: 


1. Case Study Number One 
Artist: Artist #1. 

Job Title: Forensic Photo Specialist II, full-time Composite Artist. 
Forensic Art Discipline: Composite Artist and 2D/3D Facial 
Reconstruction. 

Medium: Hand-drawn composite images. 

Training: Almost no formal training in art or interview techniques. This 
artist is self-trained, has attended several seminars, and has read 
extensively on the topic. 

Years of forensic art experience: This artist has been practicing general 
art skills since childhood and has practiced composite sketch artistry for 
16 years. The artist has completed over 200 composite renderings during 
this period. 

Number of Composite Sketches Utilized: 20 (See Appendix B-l through 
B-20.) 


2. Case Study Number Two 
Artist: Artist #2. 

Job Title: Volunteer Forensic Artist. 

Forensic Art Discipline: Composite Artist. 

Medium: Hand-drawn composite sketches. 

Training: This artist began drawing prior to a career in law enforcement 
and did not receive formal training in composite artistry. 

Years of forensic art experience: This artist is a retired police officer 
with 30 years of law enforcement experience. The artist has practiced 
composite artistry for almost 50 years. 

Number of composite sketches utilized: 9 (See Appendix C-l through 
C-9.) 
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Case Study Number Three 
Artist: Artist #3. 

Job Title: Information Specialist/Forensic Artist. 

Forensic Art Discipline: Composite Artist, 2D Design. 

Medium: Hand drawn composite sketches. 

Training: Life drawing and 2D Design; interview training: interviewing 
children of sexual abuse; FBI Forensic Facial Imaging. 

Years of forensic art experience: General Art Experience 25 years, 
Forensic Art 5 years. 

Number of composite sketches utilized: 8 (See Appendix D-l through 
D-8.) 


4. Case Study Number Four 
Artist: Artist #4. 

Job Title: Police Officer/Detective. 

Forensic Art Discipline: None. 

Medium: FACES 3.0 Computer Software. 

Training: Basic training provided by the Faces 3.0 computer tutorial. 
Years of forensic art experience: This artist is a certified police officer 
possessing basic training in interview and criminal investigation. The artist 
has used the Faces 3.0 composite art program for less than 2 years. 
Number of composite sketches utilized: 2 (See Appendix E-l and E-2.) 

C. DATA COLLECTION 

The data collection process followed the same format for each artist. As 
previously noted, the artist reviewed his or her forensic composite imagery portfolio to 
identify which of their cases had arrested suspects associated with them. These 
composites were submitted for enrollment into the Facial Recognition database. Because 
the suspect mug photo in the case is known to the researcher, the location of each mug 
photo within the Facial Recognition Database was noted in order to later determine how 
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deeply into the database the program had to search to locate the mug photo. The Facial 
Recognition Program reflects this depth as a percentage. Because the database searches 
for algorithmic matches, it searches first for the most similar photographic match to the 
composite, and then continues searching less likely matches. Because the researcher 
knows the location of the correct mug photo, this percentage is easily identified. 

As an example, a 90% result suggests that the mug photo is located using the 
enrolled composite in the first 10% of the database. This means that the database 
delivered 500 possible matches, one of which was the correct mug photo. In a real world 
scenario where the suspect is not known, a Facial Recognition Program administrator 
would then visually inspect the 500 mug photos looking for visual matches and ruling out 
photos of individuals who are the incorrect sex, race, or age. 

A 40% result suggests that the database delivered 60% of its total photographs 
before locating the mug photo. In this scenario, this means that the database delivered 
several million possible matches, one of which was the correct mug photo. It would not 
be reasonable for a Facial Recognition administrator to visually search through several 
million mug photos. 

Each suspect’s mug photo and the composite representation are displayed in an 
individual table along with the data collected from the enrollment in the facial 
recognition program. As previously noted, in an effort to provide a visual comparison of 
a portrait drawing to a composite drawing, the first sixteen composite art samples tested 
included a control drawing of the suspect created from the suspect’s mug photo. The 
display of the composite drawing, the mug photo, and the control or “portrait style” 
drawing allows a qualitative evaluation by the reader of the visual accuracy of the 
composite sketch compared to the accuracy of the “portrait style” control sketch. 

D. DATA ANALYSIS 

This quantitative data was analyzed and compared to the data associated with 

other drawings by the same artist, as well as other artists, and other composite mediums 

(sketches versus computer-aided images). In the first sixteen cases in which a control 

image was used, the difference in the percentage number matches between the control 
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image and the composite image allowed a comparative analysis of the difference in 
success between the two images. These analyses help determine the success with which 
forensic artistry can be used with facial recognition technology as a viable investigative 
tool to identify possible persons of interest. 

With regard to the qualitative data gathered, an examination of the artist’s general 
skill, training, and experience was noted to determine if there was a relationship between 
the level of training and experience and the level of success of enrolling the composite 
images in the Facial Recognition Program. 
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IV. ANALYSIS 


A. ANALYSIS OF THE FACT FINDING INQUIRIES CONDUCTED PRIOR 
TO THE ANALYSIS OF CASES 

Three inquiries or tests preceded the examination of cases, and were conducted to 
better understand how the facial recognition program would view composite sketches and 
how successfully the composite sketch might be utilized with the facial recognition 
technology to identify suspects. 

1. Fact Finding Inquiry Number 1: 

Determination of the IIS Facial Recognition Program’s Ability to 
Recognize a Composite Pencil “Portrait Style” Drawing as a Facial 
Representation 

When the “portrait style” drawing of the suspect’s mug photo was entered into the 
Facial Recognition Program, it was immediately recognized as a face and given a 
numeric representation for comparison to other photographs in the database. (See 
Appendix A-l.) 

2. Fact Finding Inquiry Number 2: 

Determination of the IIS Facial Recognition Program’s Ability to Identify 
the Mug Photo of a Subject Using a Pencil “Portrait Style” Drawing of the 
Subject in the Mug Photo 

Upon enrollment of the “portrait style” drawing into the Facial Recognition 
Program, the level of certainty was successfully raised to the standard threshold of 90% 
without elimination of the suspect photograph from the list of possible matches. At 90% 
the program yielded a list of 200 possible matches (Appendix A-2.) 

A visual comparison of the “portrait style” drawing with the suspect mug photo 
shows very little difference. Facial structure and facial features appear to be almost 
identical and skin tone and shading in the facial features is precise. 
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3. 


Fact Finding Inquiry Number 3 


Can the IIS Facial Recognition Program Identify Potential Investigative 
Leads Suspects Using a Composite Drawing of the Suspect? 

Upon enrollment of the composite representation into the Facial Recognition 
Program, the suspect mug photo was eliminated as a possible suspect at the 41st 
percentile. At 41% the suspect’s mug shot was among 3,333,585 other possible matching 
mug shots. (See Appendix A-3.) 

A visual comparison of the composite sketch and the mug photo qualitatively 
reveals several obvious differences between the composite sketch and the suspect mug 
photo. Skin tone and overall appearance of the composite image is very dark, suggesting 
to the viewer the suspect could be African-American, not Hispanic. In addition, facial 
hair depicted in the composite suggests a very different jaw line and lower facial 
structure. There are, however, some commonalities between the mug photo and the 
composite image. The suspect’s drawn face and pronounced cheekbones and lips seem to 
suggest similarities. 

B. ANALYSIS OF THE CASE STUDIES 

This research consists of four cases. Each case was associated with a single 
forensic composite artist and his or her composite artistry. 

1. Case Study Number One 

Case study number one consisted of 20 suspect composite images completed by a 
single artist. These composite sketches were completed at the time the crime occurred. 
When enrolled in the Facial Recognition Program, the composite drawings in case 
number one ranged between 14% and 50% level of certainty in identifying the true 
suspect mug photograph. On average, the composite sketches in case number one located 
the suspect at the 30.1% level of certainty. 

In this case, in addition to the composite images completed at the time the crime 

occurred, the first sixteen suspect composite comparisons also include sixteen control 
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“portrait style” drawings of the suspects created by the artist, utilizing the suspect mug 
photos as a guide. In the first 16 comparisons, each of the suspect’s three images (mug 
photo, portrait style control sketch drawn from the mug photo, and the original composite 
representation completed before the suspect was known) is displayed in an individual 
table along with the data collected from the enrollment into the facial recognition 
program for both the “portrait style” drawing and the composite drawing. This offers a 
visual qualitative representation of the relationship between the three representations of 
each suspect, as well as a quantitative representation of the data gathered. 

The 16 control drawings created by artist number one ranged between 40% and 
90% level of certainty in identifying the true suspect mug photograph. On average the 
control sketches in case number one located the suspect at the 74.4 % level of certainty. 
On average, the control drawings in case number one yielded a suspect identification at a 
44.3% higher success rate than the composite drawings. 

The level of success of these individual suspect comparisons is expressed in the 
form of a percentage. Table 1 displays the level of success for each of the control 
drawings and each of the composite sketches, together with the number of possible 
suspect mug photos delivered as a possible suspect match for the composite sketch. In 
addition, an average is expressed for both the total number of control sketches and 
composite sketches. 
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Table 1. Case #1 Results 


Suspect 

Number 

Control 

Sketch 

Percentage 

Composite Sketch 
Percentage 

Number of 
Results 

i 

90 

41 

3,333,585 

2 

64 

19 6 

9,440,604" 

3 

80 

27 

9,993,950 

4 

41 

50 

805,977 

5 

90 

32 

9,680,141 

6 

75 

27 

9,268,733 

7 

85 

30 

9,609,148 

8 

70 

34 

7,256,929 

9 

90 

48 

1,430,263 

10 

90 

25 

9,356,222 

11 

40 

19.4 

9,578,119 

12 

70 

21.6 

9,976,483 

13 

75 

20.5 

9,838,220 

14 

55 

25 

8,752,649 

15 

85 

36 

5,607,497 

16 

90 

48 

2,418,042 

17 

N/A 

34 

8,743,000 

18 

N/A 

20 

9,687,204 

19 

N/A 

14 

9,737,479 

20 

N/A 

30 

9,625,44 ll 

Average 

74.40% 

30.10% 



2. Case Study Number Two 

Case study number two consisted of nine suspect composite images completed by 
a single artist. These composite sketches were completed at the time the crime occurred. 
When enrolled in the Facial Recognition Program the composite drawings in case number 
two ranged between 18% and 57% level of certainty in identifying the true suspect mug 
photograph. On average, the composite sketches in case number two located the suspect 
at the 38.2% level of certainty. 
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Table 2. Case #2 Results 


Suspect 

Control 

Sketch 

Percentage 

Composite Sketch 
Percentage 

Number of 

Results 

] 

N/A 

30 

9,141,207 

2 

N/A 

18 

9,864,248 

3 

N/A 

57 

297.380 

4 

N/A 

31 

9,949,301 

5 

N/A 

45 

3,348,795 

6 

N/A 

57 

352.922 

7 

N/A 

35 

8,350,489 

8 

N/A 

27 

9,871,423 

9 

N/A 

44 

4,373.285 

Average 


38 20% 



3. Case Study Number Three 

Case study number three consisted of eight composite images. When enrolled in 
the Facial Recognition Program the composite drawings in case number three ranged 
between 47% and 19% level of certainty in identifying the true suspect mug photograph. 
On average, the composite sketches in case study number three located the suspect at the 
28.1% level of certainty. 


Table 3. Case #3 Results 


Suspect 

Control 

Sketch 

Percentage 

Composite Sketch 
Percentage 

Number of 

Results 

i 

N/A 

47 

2,039,137 

2 

N/A 

19 

9,295,728 

3 

N/A 

36 

9,377.816 

4 

N/A 

25 

9,540,682 

5 

N/A 

21 

9,567,465 

6 

N/A 

33 

9,534,779 

7 

N/A 

22 

8,893,227 

8 

N/A 

22 

9,760,121 

Average 


28.1% 
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4. 


Case Study Number Four 


Case study number four consisted of two suspect composite images completed by 
a single artist. These composite sketches were completed at the time the crime occurred. 
In this case two separate crimes occurred, and it was later learned that the same suspect 
had committed these crimes. A police detective using a computer software program 
completed these composite drawings. This is the only case in this research in which a 
software program was used to complete the composite drawings. 

When enrolled in the Facial Recognition Program the two composite drawings in 
case number four delivered a 48% and 26% level of certainty in identifying the true 
suspect mug photograph. On average, the composite sketches in case number four located 
the suspect at the 37% level of certainty. 

In this case, in addition to the composite images completed at the time the crime 
occurred, a control “portrait style” drawing of the suspect created by the artist, utilizing 
the suspect mug photos as a guide is included. Each of the suspect’s three images (mug 
photo, portrait style control sketch created from the mug photo, and the original 
composite representation completed before the suspect was known) is displayed in an 
individual table along with the data collected from the enrollment into the Facial 
Recognition Program for both the “portrait style” drawing and the composite drawing. 
This offers a visual qualitative representation of the relationship between the three 
representations of each suspect as well as a quantitative representation of the data 
gathered. 

The two composite renderings created by this artist were created at separate times, 
related to separate crimes. It was later learned that the suspect was the same in both these 
crimes. Although the composite drawings in this case are different, the suspect mug 
photo and the “portrait style” control drawing for both composite drawings are the same. 
The control drawing in this case located the suspect mug photo at a 29% level of 
certainty. The level of certainty of the control drawing in this case was 8% less than the 
level of certainty of the average of the composite drawings. 
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Table 4. Case #4 Results 


Suspect 

Control 

Sketch 

Percentage 

Composite Sketch 
Percentage 

Number of 

Results 

1 

29 

48 

1,071,564 

2 

29 

26 

9,999,083 

Average 

29% 

37% 



C. COMPARATIVE ANALYSIS OF ALL FOUR CASE STUDIES 

The four case studies used in this research yielded varied results. It is important to 
note that variables, including the ability of victims and witnesses to recall information, 
artist training and experience in art and interview techniques, the medium used to create 
the drawings, and how the Facial Recognition Program actually interprets these drawings 
all had some influence in the success of the Facial Recognition Program’s ability to 
locate suspect mug photos. The varied sample sizes per case also make an artist-to-artist 
comparison difficult to quantify. Conclusions regarding the case studies are drawn with 
these variables in mind. 

1. Comparative Analysis of All Artists’ Composite Sketch Percentages 

A comparison of the individual case composite sketch percentage averages offers 
a view of the level of success of each artist in utilizing a composite sketch to locate the 
suspect’s mug photo. In addition, a comparison of cases where control drawings were 
created offers a view of each artists’ level of success in utilizing a control sketch, drawn 
from the suspect mug photo, to locate the suspect’s mug photo. The artists’ averages 
ranged from 28.1 to 38.2 percent. 
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Table 5. Artist Comparison 


Artist 

Composite 

Sketch 

Percentage 

Control 

Sketch 

Percentage 

1 

30.1 

74-4 

2 

382 

N/A 

3 

28 1 

N/A 

4 

37 

29 

Average 

33.4 

51.7% 


2. Comparative Analysis of Individual Artist Experience and Training 
in Relation to Their Success in Utilizing a Composite Sketch to Locate 
a Suspect Mug Photo 

The artists’ training and experience ranged from almost none in the case of a 
detective recently trained to use the Faces 3.0 Composite software system to an artist 
with almost 50 years of training and experience. 


Table 6. Artist Experience and Training 


Artist 

Composite 

Sketch 

Percentage 

YEARS OF 
EXPERIENCE 

1 

30.1 

15 

2 

38.2 

50 

3 

28.1 

25 

4 

37 

2 

Average 

33.4% 
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V. RESULTS 


While investigators, creators of facial recognition programs, and forensic artists 
may have an idea about the ability to successfully merge the two disciplines of composite 
art and facial recognition programs, it is not known whether composite drawings can be 
used effectively in combination with facial recognition software programs to identify 
unknown suspects. The results of this research helps to understand how effectively 
composite forensic imagery can be used with facial recognition technology to increase 
the efficacy of forensic imagery as an investigative tool to identify possible persons of 
interest. 

In addition, these results offer some information to artists as well as facial 
recognition program creators to improve the level of success by which composite 
drawings are consumed by facial recognition programs to enable more effective and 
accurate identification of persons of interest. 

A. RESULTS OF THE PRE-CASE STUDY INQUIRIES 

1. Fact Finding Inquiry Number 1 

Determination of the IIS Facial Recognition Program’s Ability to 
Recognize a Composite Pencil “Portrait Style” Drawing as a Facial 
Representation 

A visual examination of the “portrait” drawing in comparison to the suspect mug 
shot reveals remarkable similarities. The portrait drawing is an almost exact replication of 
the suspect mug photo. Facial proportions and shading are precisely duplicated. 

Upon enrollment into the Facial Recognition Program, the drawing was 
immediately recognized as a facial image. Not only did the system recognize the 
drawing, it assigned a numeric representation to the drawing, allowing it to be compared 
against photographs within the database (See Appendix A-l.) 
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2. 


Fact Finding Inquiry Number 2 


Determination of the IIS Facial Recognition Program’s Ability to Identify 
the Mug Photo of a Subject Using a Pencil “Portrait Style” Drawing of the 
Subject in the Mug Photo 

The Facial Recognition Program can accurately identify a reasonable number of 
possible suspects, including the known suspect, utilizing a pencil medium “portrait style” 
control sketch. At the 90% search level, the program identified 200 possible mug photo 
matches to the drawing, including the true suspect mug photo. This 90% search level is 
the same search level deemed effective when using a photograph of an unknown suspect 
to search for a mug photo. Such a result is within the range deemed acceptable for a 
visual search of the photographs for a suspect match, making the search for this suspect 
using this particular portrait style drawing a viable method of locating the suspect. It is 
important to note that this is not reflective of a real world scenario in which a composite 
drawing would be used. The composite drawing would not be nearly as accurate as this 
portrait style sketch (See Appendix A-2.) 

3. Fact Finding Inquiry Number 3 

Can the IIS Facial Recognition Program Identify Potential Investigative 
Leads Suspects Using a Composite Drawing of the Suspect? 

An examination of the composite drawing and the mug shot photograph of the 
suspect revealed several similarities. The shape of the suspect’s head, hairline, jaw line, 
lips, and upper brow area bear a resemblance to the suspect. Skin tone and facial hair 
differ slightly in the composite drawing. As is customary with a composite sketch, 
several facial features are recognizably similar, yet many others are not. 

Quantitatively, the mug photo in comparison with the composite representation of 
the suspect was eliminated as a possible suspect at the 41st percentile, a full 50% sooner 
than the portrait style sketch was eliminated in the first inquiry. At 41% the suspect’s 
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mug shot was among 3,333,585 other possible matching mug shots. The database can in 
fact locate the suspect’s mug photo using a composite sketch for comparison but not at a 
level deemed effective in the context of a real-world investigation. 

B. RESULTS OF THE CASE STUDIES 

1. Case Study Number One 

The composite drawings in case number one ranged between a 14% and 50% 
level of certainty in identifying the true suspect mug photograph. This means that, in a 
best-case scenario, 50% of the database, or 805,977 photographs, would have to be 
reviewed in order to locate the suspect mug photo. In the worse case scenario, 86% of the 
database, or 9.7 million photographs, would have to be reviewed in order to locate the 
suspect mug photo. On average, the composite sketches in case number one located the 
suspect at the 30.1% level of certainty. 

The 16 control drawings created in case number one ranged between a 40% and 
90% level of certainty in identifying the true suspect mug photograph. This means that, in 
a best-case scenario, 10% of the database, or two photographs, would have to be 
reviewed in order to locate the suspect mug photo. In the worse case scenario, 60% of the 
database, or 3.3 million photographs, would have to be reviewed in order to locate the 
suspect mug photo. On average the control sketches in case number one located the 
suspect at the 74.4 % level of certainty. Both these levels of certainty are far below the 
level deemed effective for investigative purposes. 

2. Case Study Number Two 

Nine composite sketches were used in this case study. The composite drawings in 
case number one ranged between an 18% and 57% level of certainty in identifying the 
true suspect mug photograph. This means that, in a best-case scenario, 43% of the 
database, or 2,000 to 3,000 photographs, would have to be reviewed in order to locate the 
suspect mug photo. In the worse case scenario, 82% of the database, or 9.8 million 
photographs, would have to be reviewed in order to locate the suspect mug photo. On 
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average, the composite sketches in case number two located the suspect at the 38.2% 
level of certainty. This level of certainty is far below the level deemed effective in the 
context of a real-world investigation. 

3. Case Study Number Three 

The composite drawings in case number three ranged between a 21% and 47% 
level of certainty in identifying the true suspect mug photograph. This means that, in a 
best-case scenario, 53% of the database, or 2,039,137 photographs, would have to be 
reviewed in order to locate the suspect mug photo. In the worse case scenario, 79% of the 
database, or 9.6 million photographs, would have to be reviewed in order to locate the 
suspect mug photo. On average, the composite sketches in case number three located the 
suspect at a 28.1% level of certainty. This level of certainty is far below the level deemed 
effective in a real-world investigative scenario. 

4. Case Study Number Four 

This case study serves as the only example of the use of a computer generated 
composite sketch to identify a suspect mug photo in the Facial Recognition Program. 

Two composite sketches were used in this case study. As a method for providing an 
opportunity for qualitative analysis, a control drawing was created for each case and 
entered into the system. This control drawing represents the artist’s best effort to depict 
the suspect as accurately as possible, using the mug photo as the subject of the drawing. 
This control drawing provides an opportunity to view a suspect depiction that represents 
a best-case scenario of creating an accurate sketch of the suspect. 

The control drawing can be compared side by side with the composite drawing 
and the mug photo to allow a visual assessment of the accuracy of each drawing to the 
suspect mug photo. In addition, the control drawing results provide an opportunity for a 
quantitative comparison between the control drawing results and the composite drawing 
results. Because this sample only contains two composite representations, arriving at 
conclusions about the use of computer generated composite drawings in the Facial 
Recognition Program is difficult. 
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The composite drawings in case number four resulted in a 48% and 26% level of 
certainty in identifying the true suspect mug photograph. This means that, in a best-case 
scenario, 52% of the database, or 1 million photographs, would have to be reviewed in 
order to locate the suspect mug photo. In the worse case scenario, 74% of the database, or 
almost 10 million photographs, would have to be reviewed in order to locate the suspect 
mug photo. On average, the composite images in case number four located the suspect at 
the 37% level of certainty. 

The single control drawing created in case number four resulted in a 29% level of 
certainty in identifying the true suspect mug photograph. This means that, in a best-case 
scenario, 71% of the database, or 9.8 million photographs, would have to be reviewed in 
order to locate the suspect mug photo. 

This control drawing yielded a suspect identification at an 8% lower success rate 
than the artist’s composite drawings, meaning that the artist’s composites were more 
accurate than the actual control drawing created using the mug photo as a model. Overall, 
the level of certainty at which the suspect mug photo was identified is far below the 
acceptable threshold deemed effective for real world investigations. 

C. COMPARATIVE RESULTS OF ALL FOUR CASE STUDIES 

1. Comparative analysis of all cases average composite sketch 
percentage 

A comparison of the individual case composite sketch offers a view of each 
artist’s level of success in utilizing a composite sketch to locate the suspect’s mug photo. 
In addition, a qualitative comparison of cases where control drawings were created offers 
a view of each artist’s level of success in utilizing a control sketch drawn from the 
suspect mug photo to locate the suspect’s mug photo. 

The combined artists’ composite sketch percentage was 33.4%. While several 
drawings within each case study showed promise overall, on average the drawings 
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completed by these composite artists, although visually or qualitatively skilled, fall far 
short of being an effective means of identification of a suspect using the Facial 
Recognition system. 

2. Comparative Analysis of Individual Artist Experience and Training 
in Relation to Their Success in Utilizing a Composite Sketch to Locate 
a Suspect Mug Photo 

Forensic art experts agree that training in art, forensic art techniques, and 
interview techniques positively influences the artist’s ability to be successful in the 
creation of effective forensic sketches. In addition, the opportunity to practice these 
techniques brings a level of experience that enhances the artists’ abilities to improve their 
technique. In this analysis, the training and experience of each artist was examined in 
terms of their level of success. 

Given the varied and very limited sample size, it is difficult to identify 
conclusively to what extent the level of artist experience and training influences their 
success when using a facial recognition program. However, it appears that experience 
followed by training seems to influence the level of success when entering hand drawn 
composite sketches into the program. The artists in cases 1 and 2 have had significantly 
more experience in composite sketch artistry, and their success level suggests that this 
experience may be a contributing factor. 
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VI. CONCLUSION 


Government and law enforcement entities have invested billions of dollars into 
technologies that enable us to identify would-be terrorists and criminals through 
networked camera systems and facial recognition technologies. These programs enable 
law enforcement practitioners to identify threats and mitigate them in order to keep our 
communities safe. This research introduced a significant opportunity to expand the ability 
to identify criminals through the use of composite images in facial recognition databases 
when no photographic representation exists of the individual committing the crime. 

This research served to evaluate the feasibility of merging two previously separate 
identification technologies. Facial recognition programs are abundant, varied, and utilized 
by numerous law enforcement agencies to identify criminal suspects captured by video or 
still photography. However, it was not known whether this technology was a viable 
method to identify criminal suspects depicted in forensic composite images. The 
evaluation of a set of cases integrating composite art into facial recognition technology 
provided an opportunity to explore the possibility of expanding the suspect identification 
capabilities. 

Specifically, this research analyzed how composite forensic imagery could be 
used with facial recognition technology to increase the efficacy of forensic imagery as an 
investigative tool to identify persons of interest. The research also served as an 
opportunity for composite artists and facial recognition program creators to learn how 
they can improve the level of success by which composite drawings are consumed by 
facial recognition programs more effectively and accurately. 

A. OVERALL EFFICACY 

This research provides important insights regarding the efficacy of utilizing 
forensic illustration in a facial recognition environment. The facial recognition program 
recognizes a composite sketch as a facial image and accurately assigns a numeric 
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representation to the sketch, just as it would to a photograph. In addition, using a “portrait 
style” control sketch of the suspect’s mug photo, the program can efficiently link the 
control sketch to the mug photo at a high level of accuracy. 

These case studies focus on the ability of the Facial Recognition Program to 
effectively identify a suspect using a composite drawing. While the ability to locate the 
suspect’s mug photo in the database using a composite sketch shows promise, in order to 
be an effective method of investigation, the composite sketch would need to be searched 
at the 90th percentile and still retain the suspect mug photo in its list of possible suspects. 
Although the merging of forensic illustration with facial recognition technology is a 
viable opportunity to further criminal investigations and identify suspects, utilizing 
current systems does not yield results at a level deemed productive by practitioners. 

In order to understand why the program is unable to use the composite 
representation to effectively recognize the suspect’s mug photo, it is important to recall 
how the facial recognition program identifies possible suspect matches. Every face has 
numerous landmarks, or peaks and valleys, which make up facial features. Much like 
locations of identification on a fingerprint, these facial landmarks, known as modal 
points, are measured, creating a numerical code. Each human face has approximately 80 
modal points: for example, the distance between the eyes, the width of the nose, the depth 
of the eye sockets, the shape of the cheekbones, and the length of the jaw line. The 
program creates a numeric representation of this face using these precisely measured and 
located landmarks. 

Conversely, the creation of composite imagery is not a precise process. 

Composite art is defined as a graphic image made up from the combination of 
individually described component parts (Taylor, 2001). Composite sketches by nature are 
not intended to be precise representations. Psychologists have found that people are not 
very good at recalling a face after just a single encounter. Accuracy diminishes if the 
witness has to describe a face so that someone else can draw a picture. This is because the 
witness is asked to pick out features one by one—eyes, nose, mouth, and so on - so the 
face can be constructed (Avril & Graham, 2011). Research reveals that our brains do not 
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perceive a face in this way—like a laundry list of parts—but as a coherent whole. This 
combination of lack of precision and difficulty in recalling and describing facial features 
does not lend itself to the creation of a precise or “portrait style” composite drawing. 

While qualitatively a human being may be able to look at a composite sketch and 
recognize key features that resemble a person known to the viewer, a computer program 
does not have the ability to recognize these nuances. The facial recognition program 
relies specifically on a precise relationship between facial features in order to deliver 
possible matches. Composite images simply do not depict these facial relationships 
precisely. 

B. LESSONS LEARNED FOR ARTISTS AND FACIAL RECOGNITION 
PROGRAM DEVELOPERS 

This research provides composite artists with information to assist them in 
creating images that are more precisely consumed by current facial recognition programs, 
improving the program’s ability to identify possible investigative leads. Much of this 
information was learned through the interactive process and communication between the 
artists and the facial recognition program manager as identified in the cases. Through 
input from the program manager, images were drawn more clearly to add contrast and 
definition to the drawings, increasing the software’s ability to recognize details. 

Sketches created with less definition were not as successfully enrolled in the 
program as those that were drawn with contrast, clear lines, and obvious shading. In 
addition, composite drawings containing suspects depicted with hats had to be modified 
to remove the headwear. This headwear caused problems with the program’s ability to 
distinguish a facial feature from the headwear. While this information was beneficial for 
the consumption of composite images for the purpose of this research, these adjustments 
do very little on a broad scale to improve the ability of current facial recognition 
technology to recognize composite images. 

This research reveals two key pieces of information regarding composite art and 
its compatibility with photo facial recognition technology. The first is that composite 
images are not created with the intent to draw a portrait of the unknown suspect, and the 
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second is that there are numerous artistic styles used to create composite images. The 
mere fact that composite images are used to identify unknown suspects and that facial 
recognition programs are used to identify unknown suspects does not necessarily mean 
that the two disciplines are directly compatible in terms of integrating the two 
technologies, as they exist today. 

Photographs of unknown suspects remain fairly constant in their composition, 
clarity, detail, and accuracy, and photograph-to-photograph comparisons in a facial 
recognition program are made using the same technological language. This is not the case 
when comparing composite sketches to photographs. 

Composite images are created to highlight the noticeable and unique facial 
features of an unknown suspect. These unique facial features are showcased to trigger 
recognition in the human observer, who is capable of inference, recognition of finite 
detail, and the ability to draw conclusions based on their discretionary observations. 
Facial recognition programs do not possess these cognitive and discretionary skills; 
rather, their recognition of possible suspect matches is based solely on geometric and 
mathematical formulae, making them incapable of recognition of nuanced or inferred 
similarities between a composite image and a photograph. In short, the creation and 
viewing of composite images by human beings is subjective and open to interpretation. 

Adding to this subjectivity is the fact that composite artists create composite 
images using various artistic styles. Some depictions are lightly shaded and suggestive of 
very basic artistic lines and shapes, while others are created with definition, contrast, and 
very well-defined lines and detail. This continuum of artistic style and clarity is far from 
the uniform appearance of photographic representations. Future research into these styles 
and how they are consumed by a facial recognition program may help program designers 
build a program that better understands this format, one that is very different from one 
that analyzes photographs. Continuing these case studies with a much larger sample size 
would be beneficial in the process to identify artistic styles and continue to learn how the 
Facial Recognition Program interprets these varied styles. 
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This research is likely more beneficial to facial recognition program designers 
than it is to composite artists. While there is little that composite artists can do on a large 
scale to improve the consumption of their art by these programs, this research provides 
facial recognition program designers with information that will assist in possible design 
changes that better support the use of composite artistry in their programs in the future. 
Current software programs rely on specific facial relationships, which contribute to the 
creation of a precise algorithmic representation. It is this very precision that makes 
consumption of composite images so difficult for these programs. 

Instead of focusing on facial relationships, a program that focuses more on 
general facial features such as lines, shading, and overall facial shape to create the 
algorithmic representation may be more successful in matching mug photos to composite 
images. Michigan State University researchers are currently trying to develop a software 
program that reads these facial details in this way (Homeland Security Newswire, 2011). 
Software that finds high-level features, such as the structural distribution and the shape of 
the eyes, nose, and chin, from both the sketch and the photo would likely be more 
successful in matching composite sketches to mug photos. Currently the Michigan State 
research has only been able to perfect its software to identify the correct person 45% of 
the time. This level of success is still far below the level deemed effective in the context 
of a real-world investigation. 

C. CHALLENGES AND LIMITATIONS OF THE RESEARCH 

This research set out to determine the efficacy of using composite sketches in a 
photo facial recognition environment. In attempting to evaluate the effectiveness of such 
an idea, multiple artists were used, each with varied sample sizes, experience, training, 
and artistic style. These variables limited the ability to make comparisons from case to 
case without first acknowledging the differences in each case sample. 

This research was limited in sample size as well. The IIS Facial Recognition 
Program is used and populated by data sources specific to the state of Arizona (with some 
federal data source exceptions). This required that the composite samples come from 
Arizona cases only, somewhat limiting the overall sample size. In addition, because it 


65 



was necessary to know the identity of the suspect, only composite representations with 
identified suspects could be used, further limiting the sample size. In addition, very few 
samples utilizing composite art software programs were available for this research. While 
many Arizona agencies utilize this technology, none reported cases in which suspects had 
been identified. 

The four case studies used in this research yielded varied results. It is important to 
note that variables, including the ability of victims and witnesses to recall information, 
artist training and experience in art and interview techniques, the medium used to create 
the drawings, and the process by which the Facial Recognition Program actually 
interprets these drawings all had some influence in the success of the Facial Recognition 
Program’s ability to locate suspect mug photos. Varied sample sizes per case also made 
an artist-to-artist comparison difficult to quantify. Conclusions regarding the case studies 
were drawn with these variables in mind. 

D. REMAINING QUESTIONS OR ISSUES 

Due to the very small sample size of electronically created composite images 
using a composite art software program; this research was unable to evaluate the 
admission of electronically created composite drawings into the Facial Recognition 
Program. Increasing the sample size in future research to include more electronically 
created composite images would enable a better evaluation of the use of these composites 
in a photo facial recognition environment. 

E. CONCLUSION 

These research findings suggest an opportunity to explore the creation of a 
different type of facial recognition program that may consume composite images more 
effectively. Clearly, current photo facial recognition technology is already capable of 
recognizing a composite image as a facial representation—but not in the form that 
translates well to locating the suspect’s mug photograph at an effective level using 
current facial recognition programs. Program designers have an opportunity to fine-tune 
existing technology to more effectively link composite images to suspect mug photos. 
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The success of these experiments shows great promise for expansion of the 
usefulness of composite artistry as a tool to identify suspects on a much larger scale than 
previously experienced. As our country’s efforts in homeland security continue, the 
ability of investigators to enter composite drawings into a facial recognition program 
capable of consuming composite images more effectively will allow a composite drawing 
to be exposed to 30 million possible matches, far more effectively using the composite 
drawing to find the suspect and ultimately impact crime and criminal activity. 
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